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ABSTRACT 

Data from the National Survey of America's Families (NSAF) 
in 13 states for 1997 and 1999 was used to study how welfare reform policies, 
mandated by the 1996 passage of the Personal Responsibility and Work 
Opportunity Reconciliation Act, affected employment of single mothers with 
children. The states were Alabama, California, Colorado, Florida, 
Massachusetts, Mississippi, Minnesota, New Jersey, New York, Texas, 
Washington, and Wisconsin. The study found that, in general, there were 
increases in employment consistent with national data that show steadily 
increasing employment for single women with children over this time period. 
For high school graduates, an increase of about 5 percent in currently 
employed single mothers with children and an increase of about 7 percent in 
weeks worked last year among single mothers in the group of states was 
recorded. Contrary to expectations, the study also found that individual 
state welfare-reform policies did not have a differential effect on increases 
of single mothers previously on welfare finding and keeping employment. The 
increases were about the same in all 13 states, when accounting for general 
economic trends. (Contains 13 references.) (KC) 
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Employment and Welfare Reform 
in the National Survey of America’s Families 

Introduction 

The 1996 passage of the Personal Responsibility and Work Opportunity 
Reconciliation Act (PRWORA) was the largest federal change in welfare policy in 
decades. The sweeping federal changes included replacing the old cash assistance 
program with the Temporary Assistance for Needy Families (TANF) program, making 
assistance temporary through a five-year time limit on federal benefit receipt, changing 
state funding to a block grant, increasing state program rule flexibility, and enhancing the 
emphasis on work. Because these changes were so fundamental, there is great need to 
understand their impact, particularly on the eve of the 5-year reauthorization of the act. 

Much attention to date has focused on the impacts of welfare reform on the 
employment of current and former welfare recipients. While it is undisputed that 
employment rates have increased and that states implemented more work-focused 
policies, the causal relationship between the two has been more difficult to prove. To 
date studies that have attempted to measure the extent to which welfare policies have 
impacted work and other outcomes have had somewhat mixed results. 

This paper adds to this literature by using a new data set and method to study how 
welfare reform influenced the employment of women with children. In particular, we 
analyze the impact of state work-related welfare reform policies on employment of single 
mothers with children. 

To examine this questions we use the National Survey of America’s Families 
(NSAF). The Urban Institute and Westat first collected the NSAF in 1997 at a time that 
many states were implementing welfare reform under TANF. A second cross-sectional 
survey was conducted in 1999. These data include an over sample in thirteen focal states 
to allow for more accurate state-level analysis. We focus on these thirteen states in our 
analysis. 1 We use the two waves of this survey to examine how changes in single 
mothers' employment between waves of the survey relate to changes in employment- 
related welfare policies during the same period. 

Our use of this new data adds additional information to the discussion of causal 
impacts of welfare reform on employment. Contrary to our initial expectations, we find 
similar positive relative employment increases for unmarried women with children across 
states whether grouped by timing of policy change or type of policy change. These results 
suggest that for this early time period after reform, specific state policies did not have 
differential impact on the employment of single women with children. The final section 
of the paper discusses these results. 



1 The thirteen states are Alabama, California, Colorado, Florida, Massachusetts, Michigan, Minnesota, 
Mississippi, New Jersey, New York, Texas, Washington, and Wisconsin. 



Employment and Welfare Reform in the National Survey of America’s Families 



5 



Previous Literature 



A number of recent studies have analyzed the impact of welfare reforms on 
employment and earnings using econometric methods and secondary data sources. 2 In 
addition to the insight these studies provide on how reforms impact employment and 
other economic outcomes, we review here the methods they use and the limitations 
inherent in these methods. We discuss here several recent studies that use econometric 
methods to analyze the impact of welfare reform on employment and earnings. 3 All use 
multiple years of cross-sectional data from the Current Population Survey (CPS). 

Several studies (Meyer and Rosenbaum (2000), and Moffitt (1999)), focus solely 
on the impacts of state welfare waivers, implemented in the early to mid -990s. Since 
many state waivers had similar elements to those eventually contained in PRWORA but 
were implemented earlier, these studies are able to provide indications of PRWORA’ s 
eventual impact before much data on post-PROWRA outcomes is available. Schoeni and 
Blank (2000), Grogger (2001), and O’Neill and Hill (2001) report results for waivers and 
TANF reforms while McKeman et al. (2000) report results only for the additional effect 
ofTANF. 

In general, these studies provide evidence that both waiver and TANF reforms 
increased employment and labor supply of single mothers. Schoeni and Blank find that 
waivers significantly increased employment and weeks worked of women with less than 
a high school education. However, they do not find significant impacts ofTANF policies 
on labor force participation. Grogger finds that any reform (either a waiver or TANF) 
increased employment of femaleheaded families by 3.7 percent. O’Neill and Hill find 
that waivers increased employment of single women ages 18 to 44 by 2.3 percentage 
points and TANF increased employment for this group by 6.6 percentage points. 
Mckeman et al. find that TANF policies increased employment for single mothers with 
children by 7 to 9 percentage points. 

While there are differences in the specific models and samples used in these 
studies, they all share similar difficulties in accurately estimating the impact of reforms. 
First, welfare reform occurred in a boom economic time, so it is necessary to separate out 
economic effects from policy effects. All the studies do this by using some formulation of 
an historical time trend using data from pre-reform (and in some cases pre-economic 
boom) years. The differences in results by different specifications suggest how difficult 
it is to definitively separate these effects. Second, most of these studies rely on the 
difference in timing of implementation of waivers and TANF policies to identify effects. 
While there was variation in waiver implementation, passage of federal reform in 1996 
tended to “bunch together” many state policy changes leaving less variation. Third, full 
implementation of programs might not have yet taken place at the time of analysis. 



2 This is as compared to studies that use experimental designs to evaluate the impact of a specific policy or 
group of policies. 

3 A number of studies, including some reviewed here, focus on the impact of reform on caseload declines. 
For a review of these studies, see Bell (2001). A few studies also explore the impact of welfare reform on 
marriage and fertility. We focus here on studies analyzing employment outcomes. 
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Policies implemented with lags or not yet fully binding on behavior, such as time limits, 
make early detection of impacts difficult. 

For the most part we face these limitations in our study as well. We use a different 
methodological approach and data than these studies to try assess the differential impact 
of welfare reform across states over the same time period. We believe that if similar 
results are found with these new data and different approach, it will bolster the credibility 
of results on TANF's impact on employment. In addition, our approach allows us to 
assess whether there were differential impacts of types of policies across states, by not 
only relying on timing of policy passage, but on differences in intensity of state work- 
related policies. 



Method 

Our approach is to compare state-specific changes in employment over time for 
persons likely to be affected by welfare policy to changes for persons in the same state 
who are unlikely to be affected by welfare policy. These within-state comparisons are 
then analyzed to determine whether they vary with state welfare policies. If employment- 
related welfare policies have effects that are heavily dependent on the specifics of the 
state policies, we would expect larger employment changes in states with major policy 
changes than in other states. 

For this paper, we assume that single mothers with no more than a high school 
education are, as a group, likely to be affected by changes in welfare policy. This 
assumption will be discussed in more detail below. For each state, we calculate the 
difference across waves in average employment for single mothers. By focusing on the 
difference in employment over time within each state, we remove potential biases due to 
unobserved time-invariant state-specific factors that affect both employment of single 
mothers and the choice of state policies. 

The within-state employment change for single mothers is then compared to the 
employment change over time within state for demographic groups unlikely to be 
affected by the welfare changes. These “unaffected” or comparison groups serve to 
estimate what would have happened to employment for the affected group in the absence 
of welfare reform. The ideal comparison group would be unaffected by welfare changes, 
but would have labor market experiences quite similar to those likely to be affected by 
the change. For our analysis, we use several comparison groups, each of which is 
imperfect: married women with children, black and Hispanic males, and single women 
without children, all with no more than a high school degree. The adequacy of these 
groups is discussed in more detail below. 

The focus of our analysis is the difference-in-differences, which is the difference 
between the changes in employment over time between the affected and unaffected 
groups. The difference-in-difference is calculated within each state. It should remove 
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state-specific changes in the labor market that vary over time but are not the result of 
welfare reform. 

The difference-in-differences are then examined for patterns that might show a 
relationship to employment-related welfare policy changes. We categorize states into 
those with large and small changes in work-related welfare policies between the two 
waves of survey data. In particular, we expect larger effects in states with major 
between-wave work-related policy changes as compared with states without major 
changes. To examine this, we compare the average of the state difference-in-differences 
with those that have differing changes in policies. 4 

Rationale for our approach 

The argument for this approach is straightforward. We are interested in the effect 
of policies on the subgroup of the population most likely to be affected by the program. 
However, we have reason to believe that state-specific factors affect both labor-market 
outcomes for single mothers and policies chosen by the state. Examples of such state- 
specific factors include local preferences regarding the roles of single mothers in the 
labor market and of government. For that reason, we focus on the change in 
employment rates within a state for those affected by the policies, and examine how the 
changes are related to the change in policy within the state. Because the local labor 
market may be changing over time, we compare the change in employment rates within a 
state to the change for a comparison group assumed not to be affected by the policy. 

Another way to think about the same model is that we are interested in the effects 
of changes in policies. Because of our uncertainty about the path by which changes in 
policy have their effects, we attempt to estimate the effect of policy change within a state 
and then aggregate across states with similar changes in policy. For each state, we 
examine the change over time in employment among those likely to be affected by the 
program, and then compare this change over time to the change for those likely to be 
unaffected by the program. The differences in these changes are then averaged over 
states with comparable changes in policy. 

In practice, changes in employment for the affected and comparison groups are 
estimated using an ordinary least squares regression framework to control for changes in 
the demographic composition of the sample within each state between waves. Separate 
regressions are estimated for single mothers and for each of the various comparison 
groups, using data from two waves of the survey. The probability of employment for 
person i, living in state s in wave w, is modeled as: 

Pr obiemployment^) = Xj(5 J + <5/ + <p J s xWave 2 + 



4 We use a straight average of the state difference-in-differences. We explored use of a weighted average 
based on the size of the effected state population, but decided this gave undue weight to states with large 
populations in trying to measure effects of different states’ policies. 
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The index j indicates whether the person belongs to the affected group or one of the 
comparison groups. The variables control for changes in the composition of the 
sample over time. The controls include characteristics of the individual (age, years of 
education completed, race, and ethnicity) and the family (presence and number of 
children, age of youngest child, presence of partner); capacity to leave welfare (health 
status, access to SSI, presence of other earnings); and metro status. A given control 
variable is included in a model only if it varies within the subgroup of the population 
used for that model. 

Within each state, we estimate how employment rates change for the affected 
group and an unaffected group by comparing the state- specific components within each 
model. The model for group j contains a group-specific control for state (8^) and a 
group -specific estimate of the change in the state-component between waves (cp^). 
Estimates of the impact of programmatic changes within a state j directed at the affected 
group are estimated using the difference in the (p's between the affected and comparison 
groups (cp a s . <p c s )- We can then examine how these difference- in-difference estimates vary 
with the characteristics of the programmatic changes within each state. 

Defining Affected and Comparison Groups 

An important decision in this study is defining the affected group and an 
appropriate comparison group. We follow Moffitt (1999) and Schoeni and Blank (2000) 
in using single mothers with relatively little education as our affected group. 

Specifically, we use single mothers ages 18 to 54 with education less than or equal to a 
high school degree or GED. We expect policy changes to affect outcomes for those off 
the program as well as on the program. Those who are not currently receiving welfare 
may change their behavior as their expectations about future receipt of welfare and the 
connected requirements change. We therefore want to use a broader definition than 
simply those currently on welfare to capture these broader effects. However, it is likely 
that behavioral effects will diminish as broader populations (who are less likely to be 
eligible for welfare benefits now or in the future) are included. Therefore we limit our 
analysis to single mothers with less education. Our estimates will be an average of the 
effects across these different groups. 5 In addition, if we used a much narrower group, 
such as those on welfare, we would expect the composition of the group to be affected by 
policy directly, in turn masking part of the treatment effect. At the time of the NSAF 
interview in 1999, 15 percent of single women with children with at most a high school 
degree or GED were receiving TANF. In 1999, of married women with children with at 
most a high school degree or GED, 1 percent were receiving TANF. 

We use three separate comparison groups: married women with children, unmarried 
women without children, and black or Hispanic men. Each group includes only persons 
18 to 54 with at most a high school degree or GED. Each comparison group is expected 
to provide a measure of the within state change in employment over time and serve as a 
counterfactual for our affected group. Unmarried women without children and black 
and/or Hispanic men are generally not expected to be affected by changes in 



5 



Therefore, we expect the impacts to be smaller than what would be observed for those on welfare. 
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welfare programs and thus provide a broad counterfactual. Married women with children 
may be somewhat affected by welfare programs, in that they may be eligible now or in 
the future, so we might expect somewhat smaller differences relative to this comparison 
group. 



None of these comparison groups will perfectly reflect the counterfactual 
experience of single mothers with children if welfare reform had not passed.. We also 
include an aggregated group consisting of all persons 18 to 54 with at most a high school 
degree or GED who are not unmarried women with children. Single women without 
children, married women with children, and black/Hispanic men with low levels of 
education obviously face a different context when making employment decisions than 
single women with children. It is important to keep in mind, however, that our design 
uses the trend in employment for these groups as the comparison, not the absolute level 
of employment. Because each of these groups has shortcomings and none of them seem 
on face to be superior, we present results using all four. Prior research suggests that use 
of multiple control groups can provide a bound on estimates of the actual treatment effect 
(Rosenbaum 1987). Thus, finding similar results across these comparison groups 
increases the credibility of the results. 6 



Data 



Our data come from the 1997 and 1999 waves of the National Survey of 
America’s Families (NSAF), conducted by the Urban Institute and Westat. The NSAF 
focuses on the economic, health, and social characteristics of adults under the age of 65 
and their families. The NSAE sample was designed to be representative of the 
noninstitutionalized civilian population under age 65 in the entire nation as well as in 13 
states. Low-income households, households with children, and households in the 13 focal 
states are disproportionately included in the sample. The survey is cross-sectional, and 
includes both a random digit dial (RDD) component and an area sample of nontelephone 
households. Overall, response rates were about 65 percent in the first wave. 

The 13 focal states provide variation in the extent and timing of welfare reform. 
For instance, Massachusetts had implemented most of its welfare reform program (under 
waivers to federal rules) prior to the collection of the first wave of the NSAF. In contrast, 
states such as New York made most program changes between waves, so that the first 
wave provides a relatively clean baseline. 

The survey provides a wealth of information, including demographic 
characteristics and detailed questions about employment. It includes a detailed roster of 



6 Another possible test of the appropriateness of these groups as comparisons for single mothers with 
children is to compare the similarity of employment trends across groups for a time prior to welfare reform. 
Analysis comparing employment trends across these groups using monthly outgoing rotation group data 
from the CPS for the years 1988 to 1992 proved inconclusive. No evidence of other trends was found. 
However, there was considerable noise in the data and it is extremely difficult to control for other non- 
welfare factors that might be differentially affecting these groups in this earlier time period. 
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members of the household and their relationships, which allows us to determine who is, a 
single mother, as well as age. Questions on education provide data on most recent 
education, including receipt of GED. 

For our outcome, we focus on two measures of employment. The first is current 
employment, defined as being employed for at least an hour at the time of the survey. 

The second is the percentage of weeks worked in the year prior to the interview. Current 
employment is expected to be more random, since it depends on the week when the 
survey is asked; the weeks-worked measure is based on the entire year so variability 
should be smoothed out. However, current employment should be subject to less 
measurement error, since respondents are more likely to know their current employment 
status than to know a particular measure of information for the previous year. 

Using the two employment measures adds variability in the timing of the period 
between waves, and consequently, in which states had changes in policy. Current 
employment is measured for most of the sample between February and July of 1997 and 
1999. For this outcome measure, we consider policy changes made during the second 
half of 1997 or in 1998. Weeks worked is measured for 1996 and 1998. When using this 
outcome measure we consider policy changes that occur in calendar year 1997. Because 
of this difference in timing, we obtain a different set of states that had policy changes 
between the two waves of NSAF, one for each outcome measure. The level of 
employment for each of these measures, averaged across the thirteen focal states, is 
shown in table 1 for 1997 and 1999. 

Categorizing States 

We assigned an indicator to each state on whether major change in work-related 
welfare policies had occurred during the relevant time period. We focused on the 
following areas of work-related policy changes: time limits, work activity requirements, 
work activity sanctions, and earnings disregards. States were placed into one of three 
groups according to whether there was (1) no change between waves; (2) definite change 
between waves; and (3) somewhat mixed. The last category generally meant that some 
“major” change occurred during the collection period for the first wave of data. 

Several sources of information were used in creating these indicators according to 
the timing of policy changes. We used information from published sources, including the 
US Department of Health and Human Services report, “Setting the Baseline: A Report on 
State Welfare Waivers” (1997), Gallagher et al. “One Year After Federal Welfare 
Reform: A Description of State TANF Decisions as of October 1997”, and Urban 
Institute reports for each of the 13 focal states (available on the Urban 
Institute’s website). These reports were based in part on field visits to states that were 
conducted as part of the Assessing the New Federalism project at the Urban Institute. 
These sources were supplemented with information from the Urban Institute’s Welfare 
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Rules Database as well as conversations with Urban Institute staff conducting on-going 
field visits in the focal states. 7 

The indicators we arrived at are presented in the first two columns of table 2. 
Indicator 1 uses the time period 1996 to 1998 and indicator 2 uses the time period 1997 to 
1999. States listed as not having a major work-welfare policy change in the indicated 
time period include those who had completed their policy changes (largely through 
waivers) before the time period, and therefore had no additional major changes between 
the two years. The assignment of indicators to states involved a number of judgement 
calls. For instance, in several states, programs were first implemented in a portion of the 
state, followed by a statewide program. In such cases, we based our decision on whether 
the program was implemented in a majority of the state. Another judgement call was 
necessary when most policy changes had been made before the time period, but some 
additional changes were made during the time period. For example, Michigan had 
implemented most of its program prior to 1997, but then made some sanction policy 
changes during the spring of 1997. We chose to categorize Michigan as no policy change 
for the 1997 to 1999 period and somewhat mixed for the 1996 to 1998 period. 

Additional State Groupings 

In practice, this relatively straightforward approach is hard to implement cleanly. 
Largely, this is because the timing of policy changes is murky. Although federal 
legislation passed in August 1996, welfare policy change was not a one-time effort that 
leaves a clear pre- and post-period. Waivers from federal regulations were given to states 
before federal legislation passed, in some cases many years earlier. Different elements of 
state reforms under waivers were passed at different times, sometimes starting with one 
policy element or in a portion of the state, with the complete program phased in over 
several years. Even after passage of federal legislation, states varied in the timing for 
changing policy to reflect federally mandated elements and states had wide latitude to 
implement different policies under the new law, also put into place at differing times. 

In addition, actual implementation rarely happens immediately, particularly for 
program elements that require something other than simple application of rules. It is 
difficult to measure when changes are implemented; it is much easier to know when 
legislation is passed. Therefore, even when using a pre-post comparison based on 
legislation passage or timing of rule changes, actual implementation of the policy “on the 
ground” may lag in time when it takes effect. 

To address these difficulties, we examine other groupings of states. First, it may 
be that timing of policy does not matter as much as the intensity of the policy change. By 
intensity, we mean the strength of specific work policies such as severity of work-related 
sanctions, strict work requirements, and time limits shorter than the five-year federal 
requirement. States with tougher TANF work rules (and stronger work incentive 



7 Other helpful sources included the web sites of the Hudson Institute’s Welfare Policy Center and the 
Center for Law and Social Policy and Center for Budget and Policy Priorities’ State Policy Documentation 
Project. 
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policies) could have bigger changes in employment between waves of NSAF, regardless 
of whether these rules had already been in place for a year or two at the time of the first 
survey. We implement this categorization following Blank and Schmidt (2001). They 
create indicators of the strength of work incentives for states in each of four work-policy 
areas: benefit generosity, earnings disregards, sanction policy, and time limits. They then 
combine these to form one indicator of whether a state has strong, mixed, or weak work 
incentives. Table 2 shows this combined categorization for our 13 states under indicator 
3. 



It may also be that the date of policy passage is not the most relevant timing for 
when the policy will have impact on behavior. Following Ziliak et al. (2000), it is 
important to consider that policy may have lagged effects. For this reason, we create 
another categorization of states that indicates whether a state had pre-reform state-wide 
waivers or not. Whether major policy changes occurred between our observation periods 
is not considered to be part of this indicator. The last column of table 2 shows the 
categorization of states for this fourth indicator. Since some waivers were implemented 
in 1996, we only use this indicator to examine the change in current employment between 
1997 and 1999. 

All of these indicators try to capture different dimensions of how policy changes 
might affect employment. However, it is possible that even in states without major 
policy changes in the time period, without intense work-related policy changes, or 
without waivers, the passage of federal reform may have had an impact independent of 
the timing of state-specific policy changes. For example, the intense media coverage of 
welfare reform passage and the emphasis on work requirements may have affected 
employment outcomes in areas that did not introduce increased work requirements until a 
year or more later. Our categorization by the timing of state-specific policy changes 
cannot distinguish the potential effects of broad information at the federal level from 
those of specific policy changes. 



Results 

Before examining differences by our state categories, we present the changes in 
employment between waves of the NSAF across all of our 13 focal states. Chart 1 shows 
the unadjusted change in current employment and percent of weeks worked for unmarried 
women with children who have less than or equal to HS/GED for each of our thirteen 
states. In general, there are increases in employment or little change. Increases in 
employment are consistent with national results from the CPS that shows steadily 
increasing employment for single women with children over this time period (Lerman 
2001 ). 



When we combine all of our 1 3 within-state estimates, after controlling for other 
factors, we can observe the overall change in employment for our 13 states. Table 3 
shows the average change in current and previous year’s employment for our affected 
and comparison groups. In the first row of the first two columns, we see that unmarried 
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women with children who have less than or equal to HS/GED had a significant increase 
in employment over time using either measure. We find a 5.7 percentage point increase 
in current employment and a 6.6 percentage point increase in the percent of weeks 
worked last year. 

The remaining six columns show changes in employment across the 1 3 states for 
our comparison groups. We can see that changes in employment for the comparison 
groups by both measures are for the most part either negative or small and insignificant. 
This suggests that across all 13 states, employment for these groups was not growing 
over this time period. 8 Altogether, the consistency of the patterns leads us to conclude 
that there is growth in employment for single mothers relative to the comparison groups 
during the time between surveys. Tests of the actual difference-in-difference estimates, 
shown in the last two rows of table 3, show that this relative difference in employment 
change is statistically significant for both measures for all groups. 

Since this method controls for within-state economy-wide differences, these 
results are consistent with a significant impact of TANF policy on employment of single 
mothers with children. However, there are two caveats to this conclusion. First, this 
depends on the four comparison groups providing an accurate counterfactual for single 
women with children. Second, even if the economy effects have been separated out, this 
method does not rule out other contextual changes (such as other non-TANF policies) 
that would affect single mothers with children and not the comparison groups. TANF 
policy is obviously the most prominent possibility for this time period. 

Now we turn to examining how results vary across groups of states by the timing 
of their welfare-work-related state policy changes. We look to see if the differences are 
larger for states with major changes between our measured outcomes as compared to 
states without major changes in these time periods. All reported results for the rest of the 
paper are estimates after controlling for other factors in a regression framework as 
described earlier. 

First, we examine the changes in employment for the affected and comparison 
groups in states with major changes in work-related policies (first two rows of table 4). 
We see a significant change of about 5 percentage points in currently employed and 7 
percentage points in weeks worked last year among single mothers in this group of states. 
The estimates of change in current employment for the comparison groups are all small 
and insignificant. For weeks worked last year, two of our comparison groups show 
significant positive employment growth, married women with children and the combined 
group of all 18- to 54-year olds with at most high school education or GED. 

Employment growth for both of these groups is lower than for our affected group, 3 
percentage points and 1 percentage point, respectively. 



8 An examination of the underlying data shows that the drop in employment for unmarried mothers without 
children (which is statistically insignificant) results largely from a large negative result for the state of New 
York. 
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The difference-in-difference estimates based on these changes (shown in the next 
two rows of table 4) show a significant positive effect of welfare policy changes on 
weeks worked for three of the four comparison groups, and on current employment for 
two out of four of the comparison groups. These results suggest that states’ work-related 
welfare policy changes between the NSAF interviews had a positive impact on 
employment. 

We also examined employment changes for the group of states that we categorized as not 
having major work-related welfare policy changes over this time. Unfortunately, out of 
the 13 states, only a few states do not have a major welfare-work policy changes — only 
one state (Massachusetts) has no change between 1996 and 1998 and only three states 
(Massachusetts, Michigan, Texas) have no change between 1997 and 1999. Therefore in 
table 5, we show results only for current employment using the three states. 

For these states with no major changes in work-welfare policy between 1997 and 
1999, we find a significant increase in employment for unmarried women with children 
of 8.6 percentage points. In addition, the comparison groups show very small or 
negative coefficients, none of which are statistically significant. The difference-in- 
difference estimates reflect these results, showing significant positive relative 
employment gains for three out of the four comparison groups. Although we only have a 
subset of states, this analysis suggests that we are as likely to see positive relative 
changes in employment in states with and without major policy changes in this period. 9 

Given these results contrary to original expectations and our earlier discussion 
about the difficulties surrounding timing of policy, we examine other potential groupings 
of states to see if they show a relationship between program changes and observed 
changes. We explore two hypotheses in turn. 

First, as discussed earlier, it may be that timing of policy does not matter as much as the 
intensity of the policy change. Using indicator 3, which reflects states’ intensity of work- 
related policy change, we find that states with somewhat tougher work policies have 
similar changes in employment to those in states with more moderate policies. Table 6 
shows these results for the group of states with “strong” welfare-work policies (Florida, 
Mississippi, and New Jersey) and for those with “mixed” or “weak” policies. 10 Because 
only two of the 13 states have “weak” policies, we combine the “mixed” and “weak” 
categories together. 

In the states with strong intensity work-related policies, three out of four of the 
difference-in-difference estimates for weeks worked last year are positive and significant, 
while none of the estimates for currently employed are significant. This shows mixed 
results for whether strong intensity policies are affecting employment. However, 



9 Since we have no specific hypothesis for the “mixed change” policy states we do not discuss these results 
here. 

10 The table does not report the actual employment change estimates on which the differences- in- 
differences are based. They follow the previous tables patterns of large positive changes for unmarried 
women with children and in general small and insignificant changes for comparison groups. 
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contrary to expectations, all eight of the estimates for the states with mixed or weak 
intensity work policies are positive and statistically significant. This suggests that policy 
changes in the mixed/weak states had a similar or bigger impact on employment of single 
women with children. 

Our second hypothesis is that policies may have lagged effects, so those with pre- 
reform state -wide waivers are likely to have larger employment effects in this period 
immediately after welfare. Table 7 shows these results for the change in current 
employment. Counter to our expectations, this indicator shows similar positive and 
largely significant difference-in-difference estimates for both states with work-related 
waivers and states with no work-related waivers. In states with pre-reform waivers, three 
out of four of the comparison groups show significant positive increases in employment. 
However, in states without waivers, all of the four comparison groups show significant 
positive increases in employment. 



Discussion 

This analysis finds similar positive relative employment increases for unmarried 
women with children across states whether they are grouped by timing or type of work- 
related policy changes. We see across the board relative increases in employment, but 
little difference when we categorize states. These results are contrary to our initial 
expectations that specific state policies would have differential impact on the 
employment of single women with children. However, our overall results show 
significant positive employment changes for single women with children over and above 
the changes for other comparison groups. If these comparison groups provide a reliable 
counterfactual for general economic effects, then these results are consistent with a 
significant positive effect of TANF on employment rates. 

There are several aspects of the study data and methods that might be contributing 
to the lack of finding a differential impact across state groupings. These include the 
specific categorizations of state policies used, small number of available states, imperfect 
comparison groups, and lack of long-term historical control data. We will discuss each of 
these in turn. 

As we discussed earlier in the paper, categorizing states’ work-welfare policy 
changes is difficult for a number of reasons. Welfare reform was really a number of 
different specific policy changes possibly made at different times by states. It lacks the 
clarity of timing of a one-time federal change, such as a change in the EITC or minimum 
wage, that allows for a clear measurement of outcomes pre- and post-policy. As in many 
areas of social policy, there is also the issue that policies may be implemented on the 
ground with lags and therefore behavior may be affected with a lag. Given the nature of 
welfare reform was to give states increased flexibility, the states are really “50 
experiments” so any categorization inevitably groups together states with some 
differences in policies. It is finding the critical dimension of policy change for affecting 
behavior that is important in creating a useful categorization. In this analysis, we have 
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tried three different categorizations created along three different dimensions. However, 
there are still many other possible categorizations for just these 13 states. The fact that 
the results are so similar across these three different categorizations gives us some sense 
that other categorizations are likely to lead to similar findings. 

Another limitation of the analysis is the availability of only 13 states in the NSAF 
with sufficient data for this analysis. This limits our ability to create categorizations, 
especially multi-dimensional groupings, since in some cases only one or two of our states 
would fall into a particular category. However, no other data sets have larger state- 
specific sample sizes for a greater number of states. 

Our analysis differs from many in this literature by using new survey data from 
the NSAF for 13 states and comparing patterns of employment change for unmarried 
women with children to four reasonable comparison groups. Most of the rest of the 
literature on this topic uses the Current Population Survey, which has sufficient historical 
data for each state that state-specific time trends can be used to attempt to control for 
changes in the labor market during the period of interest. In our method, we rely on 
within-state comparison group estimates rather than relying on a state time trend for the 
counter factual. This avoids the judgement calls inherent in creating time trend controls, 
such as the appropriate length of a time trend. 

On the other hand, it puts greater weight on the importance of creating credible 
comparison groups, which is also difficult. For this reason we have relied on four 
different groups, all with similar education levels. We did not have any strong prior 
expectations on how these comparison groups might bound the true estimate. And in fact, 
we found little pattern in the results of one comparison group relative to another. 
However, we did find that each comparison group showed very similar results. By 
examining the individual group changes in addition to the difference-in-difference 
estimates, it is clear that the positive employment change is coming from unmarried 
women with children, with little significant employment change in any of the states for 
any of the comparison groups. 

In addition to these methodological issues, it is also possible that what these 
results suggest is true — there were not large differences in employment impacts across 
states with different policies in the first years after the passage of federal welfare reform. 
One possible explanation for these results is that individual behavior was affected directly 
by the large amount of publicity and intense media coverage around passage of the 
federal government’s welfare reform act in 1996. It can be argued that the message of 
welfare reform was so clearly that “now you have to work” that it affected individual 
behavior across states regardless of their specific policy implementation or timing. In 
addition, in some states, knowledge among TANF case workers that work-focused 
policies were coming although not yet implemented, led them to convey that message of 
work to recipients. 11 This possibility is also bolstered by the fact that many on-going 
experimental design studies in states with welfare waivers found significant changes in 



11 This information is based on conversations with researchers studying implementation of welfare reform 
in the states. 
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employment and welfare participation among control groups after federal reform passed. 
This suggests that there may have been misunderstandings and miscommunications 
among recipients as to whether they were covered by new policies. This same 
misunderstanding could have occurred in states that had not implemented strong work- 
policy changes. 

This does not suggest that there is no differential impact of state policies or that 
specific state welfare-to-work policies did not affect employment. It may be that in this 
early period after national passage of welfare reform legislation, differentials are small 
and hard to measure. As time since PRWORA passes and states have fully implemented 
their own work policies, we may find differential impacts of state work-welfare policies. 
However, problems of categorization of state policies and the actual timing of 
implementation will remain. In an era where there are 50 separate state experiments (or 
even more given sub-state variation) estimation of the impacts of welfare-work policy 
changes is difficult. 
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Table 6 

Difference-in-Difference Estimates of Employment by Intensity of Work-Related Welfare Policies by Demographic Group, 

Age 18 to 54 with less than or equal to HS/GED 
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definitions. “Currently employed” change is for 1997 to 1999. “Weeks worked last year” change is 1996 to 1998. 
* p<.l; ** p<05 

Source: 1997 and 1999 National Survey of America’s Families. 



Table 7 

Difference-in-Difference Estimates of Employment by State-Wide Work-Welfare Policy Waiver by Demographic Group, 

Age 18 to 54 with less than or equal to HS/GED 
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Chart 1 

Employment Changes by State for Unmarried Women 18-54 with Children 
and less than or equal HS/GED, 1997-1999 NSAF 
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